Method of generating a multi-modality anatomical atlas

ABSTRACT

Method of generating a multi-modality anatomical atlas. The method includes receiving first and second medical images of a region-of-interest (ROI) of a same individual. The first and second medical images are acquired by different first and second imaging modalities. The method includes generating first and second feature images based on the first and second medical images. The first and second feature images include a same designated anatomical feature of the ROI. The method includes determining a transformation function by registering the first and second feature images and applying the transformation function to the first and second medical images to register the medical images. The method includes generating a multi-modality anatomical atlas. The multi-modality atlas has the first and second medical images. The first and second medical images are first and second reference images. The multi-modality anatomical atlas includes an organ model that corresponds to an organ in the ROI.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a divisional of U.S. application Ser. No.13/561,717, filed on Jul. 30, 2012, which is incorporated herein byreference in its entirety.

BACKGROUND

The subject matter disclosed herein relates generally to medical imagingsystems, and more particularly, to methods and systems that facilitateregistering medical images for analysis.

Various image-processing workflows exist in which medical images mayundergo registration and segmentation operations. During a registrationoperation, medical images from different time periods and/or medicalimages acquired through different modalities are mapped together. Duringa segmentation operation, anatomical structures or features areextracted from the medical images. For example, a medical image may beanalyzed to identify pixels (or voxels) of the medical images ascorresponding to a designated organ (e.g., liver) or a designated tissue(e.g., soft tissue). The medical images may be acquired by differentimaging modalities (e.g., ultrasound; magnetic resonance imaging (MRI);computed tomography (CT); positron emission tomography (PET); singlephoton emission computed tomography (SPECT); etc.). As one example, apatient may go through a series of imaging sessions to monitor thepatient's response to treatment. The medical images obtained duringthese imaging sessions may be collected and analyzed together.

However, the registration and segmentation operations can require asignificant amount of time and/or computing resources. Furthermore, alarge number of clinical applications require specific information tofacilitate the registration and segmentation operations. In some cases,the information must be provided by the user. For example, it may benecessary for the user to identify a type of segmented lesion, to draw aseed region in a target organ to initiate a semi-automatic segmentation,or to identify a region in a target organ for additional measurements.

BRIEF DESCRIPTION

In one embodiment, a method is provided that includes receiving an inputimage of a region of interest (ROI) of an individual. The input image isa medical image acquired by a first imaging modality. The method alsoincludes generating a first feature image based on the input image. Thefirst feature image includes a designated anatomical feature of the ROI.The method also includes obtaining an anatomical atlas. The atlas has areference image of the ROI of at least one other individual and an organmodel. The reference image is a medical image that is acquired by asecond imaging modality that is different from the first imagingmodality. The method also includes determining a transformation functionby registering the first feature image with a second feature image thatis based on the reference image and includes the designated anatomicalfeature.

Optionally, the method may also include applying the transformationfunction to the organ model to generate a registered organ model andusing the registered organ model to obtain a final organ position of anorgan within the ROI. The final organ position of the organ in the ROImay be used to at least one of (a) automatically identify lesion typesbased on the final organ position; (b) generate one or more seed regionsto segment the organ; or (c) identify a representative region in theorgan and automatically calculate measurements of the organ in therepresentative region.

In another embodiment, a system is provided that includes afeature-image generator configured to analyze an input image to generatea first feature image. The input image includes a region of interest(ROI) of an individual and is a medical image acquired by a firstimaging modality. The first feature image includes a designatedanatomical feature of the ROI. The system also includes a storage systemconfigured to store an anatomical atlas. The atlas includes a referenceimage of the ROI of at least one other individual and an organ model.The reference image is a medical image acquired by a second imagingmodality that is different from the first imaging modality. The systemalso includes a transformation module that is configured to determine atransformation function by registering the first feature image with asecond feature image that is based on the reference image and includesthe designated anatomical feature.

In a further embodiment, a non-transitory computer readable medium isprovided that is programmed to instruct a computing system to receive aninput image of a region of interest (ROI) of an individual. The inputimage is a medical image acquired by a first imaging modality. Thecomputer readable medium is also programmed to instruct the computingsystem to generate a first feature image based on the input image. Thefirst feature image includes a designated anatomical feature of the ROI.The computer readable medium is also programmed to instruct thecomputing system to obtain an anatomical atlas. The atlas includes areference image of the ROI of at least one other individual and an organmodel. The reference image is a medical image acquired by a secondimaging modality that is different from the first imaging modality. Thecomputer readable medium is also programmed to instruct the computingsystem to determine a transformation function by registering the firstfeature image with a second feature image that is based on the referenceimage and includes the designated anatomical feature.

In another embodiment, a method of generating a multi-modalityanatomical atlas is provided. The method includes receiving first andsecond medical images of a region-of-interest (ROI) of a sameindividual. The first and second medical images are acquired bydifferent first and second imaging modalities. The method also includesgenerating first and second feature images based on the first and secondmedical images. The first and second feature images include a samedesignated anatomical feature of the ROI. The method also includesdetermining a transformation function by registering the first andsecond feature images and applying the transformation function to thefirst and second medical images to register the medical images. Themethod also includes generating a multi-modality anatomical atlas. Themulti-modality anatomical atlas has the first and second medical images.The first and second medical images are first and second referenceimages. The multi-modality anatomical atlas includes an organ model thatcorresponds to an organ in the ROI.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a computing system formed in accordancewith one embodiment.

FIG. 2 is a flowchart of a method for determining a spatial placement ofone or more organs in a region-of-interest (ROI) in accordance with oneembodiment.

FIG. 3 is another flowchart that further illustrates a portion of themethod shown in FIG. 2.

FIG. 4 is another flowchart that further illustrates a portion of themethod shown in FIG. 2.

FIGS. 5A and 5B illustrate a flowchart of a method of generating amulti-modality anatomical atlas in accordance with one embodiment.

FIG. 6 is a pictorial view of a multi-modality imaging system formed inaccordance with one embodiment.

FIG. 7 is a block schematic diagram of a CT portion of the systemillustrated in FIG. 6.

DETAILED DESCRIPTION

Embodiments described herein include methods, systems, and computerreadable media that may facilitate at least one of processing oranalyzing medical images of a region-of-interest (ROI) (also referred toas a volume-of-interest (VOI)). For example, embodiments may includemethods, systems, and computer readable media that generate amulti-modality anatomical atlas. Embodiments may also include methods,systems, and computer readable media that determine a spatial placementof one or more organs in a region-of-interest (ROI).

The medical images may include image data or datasets that represent avisualization of the ROI. The image data may include pixels (or voxels)having signal intensity values or other values/qualities/characteristicsthat may be processed to form the visualization. Various imagingmodalities may be used to acquire the medical images. Non-limitingexamples of such modalities include ultrasound, magnetic resonanceimaging (MRI), computed tomography (CT), positron emission tomography(PET), and single photon emission computed tomography (SPECT). Themedical images may be two-dimensional (2D), three-dimensional (3D), andfour-dimensional (4D) medical images of the ROI.

In some embodiments, the medical images are converted from one type ofmedical image (e.g., a CT image of the ROI) to a feature image. Morespecifically, the medical image may be processed to identify or extractan anatomical feature from the medical image. As used herein, an“anatomical feature” may include, for example, bone, soft tissue, fattytissue, or air that is within or surrounds an anatomical structure(e.g., lungs). By way of one example only, an embodiment may convert aCT image of an ROI into a first bone image that is representative of thebone in the ROI as acquired by a CT imaging system. The embodiment mayalso convert a PET image of the ROI into a second bone image that isrepresentative of the bone in the ROI as acquired by a PET imagingsystem. As will be described below, the first and second bone images maythen be used to calculate a transformation function for registering anorgan model(s). In some embodiments, the different medical images may befrom one patient. Feature images of the different medical images fromthe same patient may be used to calculate a transformation function forregistering the medical images or organ models. The registered imagesmay then be added to an atlas.

One or more embodiments may utilize an anatomical atlas during theprocessing of the medical image (e.g., during the registration and/orsegmentation of the medical image). As used herein, an “anatomicalatlas” may include one or more reference medical images. The referenceimages may be medical images of another individual other than theindividual-of-interest (e.g., patient). The reference images are medicalimages that are suitably representative of the ROI. In some cases, thereference images may be analyzed by a clinician, researcher, or othersuitable person and determined to be representative of the ROI. Theselected reference image may be a typical or an illustrative example ofthe ROI as imaged by a particular modality for a general population orfor a particular demographic. For example, the reference image may beselected based on the shape and relative positions of multiple organs inthe ROI for an average individual, an average adult, an average child,an average male or female adult (or child), an overweight adult, or anadult that has a history of smoking. In some embodiments, the anatomicalatlas includes a plurality of reference images in which the referenceimages are associated with different demographic groups. For instance,in one embodiment, the anatomical atlas may include three adult-malereference images; three adult-female reference images; and three childreference images. An anatomical atlas may include reference images froma plurality of different modalities for the same individual.

An anatomical atlas may also include one or more organ models, which mayalso be referred to as an organ lexicon. Organ models may includeprobabilistic data relating to one or more organs. Each organ model maybe based on a plurality of organs from different individuals. Forexample, a liver model may include probabilistic data that is based on aplurality of livers from different individuals. The probabilistic datamay relate to at least one of a shape, volume, or position of thedesignated organ in the ROI. As one example, the probabilistic data mayinclude a probability that a pixel (or group of pixels) in the imagedata corresponds to a portion of an organ. The probability may be based,at least in part, on a location of the pixel. The location of the pixelmay be identified by its position relative to an identifiable landmarkor a designated point in a coordinate system. Whether or not the pixelis assigned to, for example, an organ, soft tissue, bone, air, or otheranatomical feature may be based on the signal intensity value of thepixel and the probability provided by the organ model. In some cases,the probabilistic data is based on a plurality of medical images. Likethe reference images, the organ models may be associated with differentdemographic groups or groups for individuals with a common medicalhistory (e.g., cancer survivor, smoker, etc.).

The above and the following detailed description of various embodimentswill be better understood when read in conjunction with the appendeddrawings. To the extent that the figures illustrate diagrams of thefunctional blocks of the various embodiments, the functional blocks arenot necessarily indicative of the division between hardware circuitry.Thus, for example, one or more of the functional blocks (e.g., modules,processors, or memories) may be implemented in a single piece ofhardware (e.g., a general purpose signal processor or a block of randomaccess memory, hard disk, or the like) or multiple pieces of hardware.Similarly, the programs may be stand alone programs, may be incorporatedas subroutines in an operating system, may be functions in an installedsoftware package, and the like. It should be understood that the variousembodiments are not limited to the arrangements and instrumentalityshown in the drawings.

One or more embodiments may be implemented using an imaging agent or mayinclude medical images that were acquired during the use of an imagingagent. The term “imaging agent,” as used herein includes any and allradiopharmaceutical (RP) agents and contrast agents used in connectionwith diagnostic imaging and/or therapeutic procedures. The imaging agentmay represent a perfusion agent. The imaging agent may be, among otherthings, an imaging agent adapted for use in MRI (including functionalMRI), an intravenous CT contrast agent, a radiopharmaceutical PET orsingle photon emission computed tomography (SPECT) tracer, an ultrasoundcontrast agent, an optical contrast agent, myocardial perfusion tracers,cerebral perfusion tracer and the like. By way of example only, theimaging agent may be Myoview™, Fluorodeoxyglucose (FDG),¹⁸F-Flourobenzyl Triphenyl Phosphonium (¹⁸F-FBnTP), ¹⁸F-Flouroacetate,¹⁸F-labled myocardial perfusion tracers, Tc-ECD, Tc-HMPAO, N-13 ammonia,Envision N-13H3, Iodine-123 ligands, ⁹⁹m-Technitium ligands, Xenon-133,Neuroreceptor ligands, etc.), 18F-fluoromisonidazole, ²⁰¹Thallium,^(99m)Technetium sestamibi, and ⁸²Rubidium among others.

FIG. 1 is a block diagram of a computing system 100 formed in accordancewith one embodiment. The computing system 100 may be used to, forexample, process medical images to facilitate analyzing anatomicalregions or structures in the ROI. The computing system 100 includes asystem controller 102 and a user interface 104. The system controller102 is communicatively coupled to the user interface 104 and may also becommunicatively coupled to one or more imaging systems (e.g., CT, MR,PET, etc.).

In an exemplary embodiment, the system controller 102 includes one ormore processors/modules configured to process and, optionally, analyzethe medical images to identify areas-of-interest in an anatomicalstructure(s), such as lesions in an organ. For instance, the systemcontroller 102 may include one or more modules configured to execute aset of instructions that are stored in one or more storage elements(e.g., instructions stored on a tangible and/or non-transitory computerreadable storage medium, excluding signals) to process the medicalimages. The set of instructions may include various commands thatinstruct the system controller 102 as a processing machine to performspecific operations such as the workflows, processes, and methodsdescribed herein. By way of example, the computing system 100 may be orinclude a desktop computer, laptop, notebook, tablet computer, or smartphone.

The user interface 104 may include hardware, firmware, software, or acombination thereof that enables an individual (e.g., a user) todirectly or indirectly control operation of the system controller 102and the various components thereof. As shown, the user interface 104includes an operator display 110. The operator display 110 is configuredto display one or more images, such as the medical images and processedimages described herein. In some embodiments, the user interface 104 mayalso include one or more input devices (not shown), such as a physicalkeyboard, mouse, and/or touchpad. In an exemplary embodiment, theoperator display 110 is a touch-sensitive display (e.g., touchscreen)that can detect a presence of a touch from an operator of the computingsystem 100 and can also identify a location in the display area of thetouch. The touch may be applied by, for example, at least one of anindividual's hand, glove, stylus, or the like. As such, thetouch-sensitive display may receive inputs from the operator and alsocommunicate information to the operator.

The system controller 102 includes a plurality of modules or sub-modulesthat control operation of the system controller 102. For example, thecomputing system 100 may include modules 121-125 and a storage system126 that communicates with at least some of the modules 121-125. Themodules include a feature-image generator 121 that is configured toanalyze a medical image to generate an image (e.g., a feature image)that is based on the medical image and includes a designated anatomicalfeature. The feature-image generator 121 may analyze pixel values (e.g.,signal intensity values or other values) to identify the pixels thathave pixel values above or below a predetermined limit or within apredetermined range. The identified pixels may form the basis of a boneimage, soft tissue image, fatty tissue image, or air image. As usedherein, the term “pixels” may be used interchangeably with “voxels”unless explicitly noted to the contrary.

The modules 121-125 may also include a transformation module 122. Aswill be described in greater detail below, embodiments described hereinmay be configured to calculate a transformation function (or model) forregistering images. In particular embodiments, the transformationfunction is determined by registering an anatomical feature image fromone imaging modality with an anatomical feature image from anotherimaging modality. The modules 121-125 may also include a registrationmodule 123 that is configured to apply the transformation function to anorgan model(s) to register the organ model(s) with an input image.

The modules 121-126 may also include an atlas generator 125 that isconfigured to generate an anatomical atlas. The atlas generator 125 mayreceive and store designated medical images, including feature images,in the storage system 126 or other storage system. The atlas generator125 may designate the medical image(s) as being part of an anatomicalatlas. The atlas generator 125 may assign identifying labels or otherinformation to the medical images. For example, the information may bein accordance with established protocols (e.g., Digital Imaging andCommunications in Medicine (DICOM)). The information may also specifyanatomical information or demographic information. For example, theatlas generator 125 may designate a medical image as a CT image for aliver (or other organ) in the anatomical atlas that may be used as areference image. As will be described below, the atlas generator 125 maygenerate a multi-modality anatomical atlas. The multi-modalityanatomical atlas may include reference images and organ models that arebased on medical images from multiple modalities.

The atlas generator 125 may also generate or modify organ models. By wayof one example, embodiments described herein may update or modifyanatomical atlases. After medical images from various individuals areregistered and/or segmented, the atlas generator 125 may add the datafrom the registered and/or segmented medical images to one or more theanatomical atlases. The data may include or identify a location and/orcontour of one or more organs. The added data from the medical imagesmay affect (e.g., change) the probabilistic data of the organ models.

A graphical user interface (GUI) module 124 may coordinate with theother modules and the user interface 104 for displaying various objectsin the operator display 110. For example, various images ofuser-selectable elements may be stored in the storage system 126 andprovided to the operator display 110 by the GUI module 124. The GUImodule 124 may be configured to prompt or request user inputs from theuser of the computing system 100. For example, the GUI module 124 mayrequest a user to identify the input images to be processed or theanatomical atlas to be loaded. In some embodiments, the GUI module 124may also request the user to provide certain information. For instance,the GUI module 124 may request that the user identify or label lesionswithin an organ.

In the illustrated embodiment, the computing system 100 may be part of amulti-modality imaging system. However, in other embodiments, thecomputing system 100 is a separate and distinct system that isconfigured to receive medical images remotely from one or more imagingsystems.

FIG. 2 is a flowchart of a method 200 for determining a spatialplacement of one or more organs in the ROI in accordance with oneembodiment. Exemplary organs include the liver, heart, spleen, kidneys,and the like. At least some of the operations in the method 200 may beimplemented automatically by a computing system, such as the computingsystem 100 (FIG. 1). In FIGS. 2-4, each of the indicated operations mayautomatically be performed by a computing system. However, in someembodiments, the user may provide a request or at least some input.

The method 200 includes receiving at 202 a medical image of a ROI of anindividual. To distinguish between other images, the medical imagereceived at 202 is hereinafter referred to as the “input image.” As usedherein, the term “receiving” may include receiving the input imageremotely from a system that is outside of the computing system 100 (oroutside of a local area network to which the computing system 100belongs) or obtaining the input image from a storage system (e.g.,database) that is part of the computing system 100, such as the storagesystem 126 (FIG. 1). The receiving operation may also include receivingthe input image through a communication network or receiving the inputimage from a portable storage device (e.g., USB). By way of one specificexample, the receiving operation at 202 may occur when a user of thecomputing system 100 selects at the user interface 104 a 3D medicalimage that includes multiple organs for examination.

The input image may be acquired by a first imaging modality (e.g., firsttype of imaging modality). As described above, medical images may beacquired by one or more modalities, including ultrasound, CT, MR, PET,and SPECT. The individual may be, for example, a patient that desires adiagnosis or a patient that desires to monitor treatment. However,embodiments described herein may be also used for research purposes. Assuch, the individuals are not required to be patients. Individuals arealso not required to be human. In an exemplary embodiment, the ROI is atorso of the individual or a particular portion of the torso (e.g., theROI may be designated to at least include the lungs, liver, heart,spine, spleen, a designated skeletal structure, and the like). However,in other embodiments, the ROI is not limited to the torso and mayinclude other regions of the body, such as the legs or head.

The method 200 may also include loading at 204 an anatomical atlas intothe computing system 100. The anatomical atlas may include one or morereference images and one or more organ models. The loading operation at204 may occur when a user selects at the user interface 104 thedesignated anatomical atlas. In some embodiments, the user interface 104may present to the user a plurality of anatomical atlases to selectfrom. The anatomical atlas may be a multi-modality anatomical atlas,such as the anatomical atlases described below. In other embodiments,the loading operation at 204 may also include receiving the anatomicalatlas. For example, the anatomical atlas may be delivered with the inputimage. It is understood that the loading operation at 204 may occurafter, before, or concurrently with the receiving operation at 202.

At 206, the input image received at 202 and the anatomical atlas loadedat 204 may be registered at 206 with respect to each other. In someembodiments, the computing system 100 may examine or analyze theanatomical atlas to determine if the anatomical atlas includes asuitable reference image for the input image. For example, if the inputimage is a CT image of an individual's torso, the computing system 100may determine if the anatomical atlas includes an image that is a CTimage of another individual's torso. As one specific example, each ofthe input image and the atlas image(s) may be assigned or labeled withidentifying information (e.g., DICOMM information). The computing system100 may compare the identifying information of the input image and theatlas image(s) to determine if one of the atlas images may be used as areference image. If the computing system 100 determines that an atlasimage may be used as a reference image, then the registering operationat 206 may use the designated reference image.

However, as will be described in reference to FIG. 3, the registrationoperation at 206 may include registering a feature image that is basedon the input image of a first imaging modality with a common featureimage of a second imaging modality. After the registration at 206, aspatial placement of an organ (or spatial placements of multiple organs)is determined at 208. As used herein, a “spatial placement” may includea position (e.g., location and orientation) of the organ(s) within a 2Darea or 3D space. The spatial placement may also include a shape of theorgan within the 2D area or 3D space.

After the spatial placement has been determined at 208, the processedimage data may then be used for one or more applications in which acalculated spatial placement of the organ is desired. For example, someembodiments may use the image data obtained to perform automaticorgan-specific lesion labeling. The computing system 100 may analyze theimaged organ(s) to automatically identify a number of suspected lesionsin the organ(s). Suspected lesions may include abnormal structures orgrowths (e.g., tumors) within the organ. The suspected lesions maycorrespond to areas of the image that have unexpected light intensityvalues. Another example may include automatically segmentingorgans-of-interest based on the final organ position. Moreover, thecomputing system 100 may identify a representative region of the organbased on the final position and calculate desired statistics of theorgan from the representative region.

FIG. 3 illustrates the registration at 206 in accordance with oneembodiment. As described above, the registration at 206 may includeidentifying a reference image from the anatomical atlas. If a referenceimage is identified, then the input image and the identified referenceimage may be used to generate a transformation function as describedbelow. However, in some embodiments, the registration at 206 may includegenerating at 210 a first feature image that is based on the inputimage. For example, the input image may be acquired from a CT imagingsystem and the designated anatomical feature may be soft tissue or bone.The generation at 210 may include analyzing the input image andprocessing the input image according to a designated algorithm orprotocol to generate the first feature image. The algorithm may, forexample, analyze a signal intensity for each pixel (or voxel) in theinput image. For each pixel (or voxel) that has a signal intensity aboveor below a designated limit or within a designated value range, thepixel (or voxel) may be assigned or labeled as being a feature pixel (orvoxel). Using the above example, the identified pixels may be part of a“soft tissue image.” When viewed, the soft tissue image may visualize anapproximate soft tissue structure(s) of the input image.

In other embodiments, the designated algorithm or protocol to generatethe first feature image at 210 may include region growing. For instance,the computing system 100 may select a point within a region of the inputimage. The selected point may have a high probability of being locatedwithin a predetermined anatomical structure, such as an organ (e.g.,liver). The algorithm may analyze pixels that neighbor (e.g., areadjacent to) the selected point and determine whether the neighboringpixels satisfy a predetermined criteria (e.g., within a predeterminedsignal intensity range). In this manner, regions of the image that aresimilar to each other are aggregated together in the feature image and,as such, may indicate a designated anatomical feature.

The registration at 206 may also include obtaining at 212 a secondfeature image from the anatomical atlas loaded at 204. The secondfeature image may be based on another medical image that is acquiredfrom a second imaging modality that is different than the first imagingmodality. As one example, when the first imaging modality is CT, thesecond imaging modality may be PET or MR. However, the second featureimage may include the same designated anatomical feature as the firstfeature image. For instance, each of the first and second feature imagesmay be a soft tissue image.

In some embodiments, the anatomical atlas includes one or more referenceimages that were acquired using the second imaging modality. Thereference images may be images of other individuals that are designatedas representative images of the ROI. In such embodiments, theregistration 206 may include selecting a reference image at 214 from theplurality of reference images and generating at 216 a second featureimage based on the selected reference image. The selected referenceimage may be selected by comparing identifying information (e.g., DICOMMinformation) of the input image to the potential reference images. Forinstance, the input image may be a CT image of a torso and the referenceimage may be a PET image of a torso. The selected reference image mayalso be selected by finding a reference image having an approximate sizeand shape of an organ-of-interest as the input image or by demographicfactors. The generation at 216 may be similar to the generation at 210and include analyzing the reference image and processing the referenceimage according to a designated algorithm or protocol to generate thesecond feature image, such as those described above with respect to thegeneration at 210.

In other embodiments, the anatomical atlas may include a one or moresecond feature images. More specifically, the anatomical atlas mayinclude a second feature images that are based on reference images. Insuch embodiments, the obtaining at 212 may include selecting at 218 asecond feature image from second feature images that werepreviously-generated. As described above, the selected second featureimage may be selected by comparing identifying information of the secondfeature images and the input image. The selected second feature imagemay also be selected by finding a second feature image having anapproximate size and shape of an organ-of-interest as the input image orby demographic factors.

At 220, one or more identifiable portions or sections of the first andsecond feature images are determined. The identifiable portions may be,for example, recognizable portions of the feature images that correspondto anatomical structures of the ROI. The identifiable portions may be,for example, particular cross-sections of an organ (e.g., liver) orother dimensions of an organ, internal landmarks (e.g., particular bonestructures or non-anatomical markers inserted into patient), surfacecontours, external non-anatomical markers (e.g., stickers orstereotactic frames), and the like. By way of one example, if the firstand second feature images include the liver, then the determiningoperation 220 may include identifying the largest cross-section of theliver in each of the feature images. More than one identifiable portionin the first and second feature images may be included. For example,multiple cross-sections of a single organ may be used or cross-sectionsof different structures may be used.

The method may also include determining at 222 a transformation functionby registering the first and second feature images. The determining at222 may be based on the identifiable portions determined at 220. Thetransformation function indicates the degrees of freedom by which one ofthe feature images can be deformed to match the other feature image.More specifically, the transformation function may indicate how much thesecond feature image should be rotated, translated, scaled, and/orsheered to be registered with the first feature image.

The determination at 222 may include rigid transformation at 224 and/ornon-rigid transformation at 226. In the rigid transformation at 224, theidentifiable portion(s) that were determined at 220 may be at least oneof translated or rotated about each one of the x-, y-, and z-axes untilthe identifiable portions of the first and second feature images arealigned. The rotation and/or translation about each axis may becalculated and be a part of the transformation function.

In some embodiments, the first and second feature image may also undergothe non-rigid transformation at 226. The non-rigid transformation at 226may include scaling and/or sheering one of the images so that the firstand second feature images overlap each other. For example, if anorgan-of-interest in the first feature image has a similar shape but adifferent size than the organ-of-interest in the second feature image,then the second feature image may be scaled until the organs-of-interestin the first and second feature images have a similar size. In somecases, the scaling may also achieve more overlap along, for example, thesurface contours of the organs-of-interest. The amount of scaling and/orsheering may be calculated and be part of the transformation function.The transformation function is determined (e.g., calculated) by mergingor combining the calculations made during the rigid and non-rigidtransformations.

At 228, the transformation function is applied to at least one of theorgan models of the anatomical atlas to provide registered organ models.In some cases, the organ model(s) may be associated with the secondfeature image. For example, if the second feature image was derived froma reference image of the anatomical atlas, the organ model used duringthe application at 228 may be the organ model associated with thereference image. The application at 228 may be characterized astransferring the organ model(s) to a coordinate system of the inputimage.

However, as described above, the registration operation at 206 (FIG. 2)may include identifying a reference image of the same imaging modalityfrom an anatomical atlas. For example, if the input image is a PET imageof a torso of an individual, the registration operation at 206 mayinclude identifying a reference image a PET image of a torso in theanatomical atlas. In such embodiments, the input image and the referenceimage of the same imaging modality may then be registered and atransformation function may be determined in a similar manner asdescribed above with respect to the first and second feature images. Forexample, identifiable portions in the input image and the referenceimage may be determined as described above with respect to thedetermining operation at 220. Like the determining operation at 222, atransformation function may then be determined by registering the inputand reference images of the same imaging modality. The determination mayinclude rigid and non-rigid transformations as described above.Furthermore, the determined transformation function may then be appliedto the one or more organ models of the anatomical atlas. The organmodels may then be registered to the input image.

FIG. 4 illustrates the determining at 208 of the spatial placement ofthe one or more organs in the input image in greater detail. Inparticular embodiments, the determining at 208 includes identifying acorresponding position for each one of a plurality of organs in the ROI.As described above, registered organ models may be determined at 228. At208, the registered organ models may be applied to the organs in theinput image.

The determining at 208 may include applying the same algorithm ordifferent algorithms to the input image and the registered organ models.In an exemplary embodiment, the determining at 208 includes applying at230 an intensity-based threshold to the input image to obtain asegmented input image. The thresholds may differ based on theorgan-of-interest (e.g., liver, heart, spleen, etc.) and the type ofinput image (e.g., CT, MR, PET, etc.). The determining at 208 may alsoinclude applying at 232 probability-based thresholds to the registeredorgan models to obtain a segmented organ model. The probability-basedthresholds may differ based on the organ-of-interest and the type ofinput image.

At 234, organ placements (e.g., position, orientation, and contour) areapproximately identified based on an intersection of the segmented inputimage and organ model(s). At 236, unwanted artifacts or noise may beremoved through post-processing thereby determining final organplacements in the ROI. After the organ placements are identified, theoperator display 110 may display an image of the ROI that includes oneor more organs having the determined spatial placement. When more thanone organ is included in the image, the organs may be shown relative toeach other. In some embodiments, data corresponding to an organplacement of an organ may be added to an anatomical atlas. For instance,data that describes at least one of a shape (or contour), volume,position, or orientation of the organ may be added to the anatomicalatlas. The added data may be considered with other organ data (e.g.,other data that describes organ placement) to determine theprobabilistic data of the organ models.

FIGS. 5A and 5B illustrate a flowchart of a method 260 of generating amulti-modality anatomical atlas in accordance with one embodiment. Aftergeneration, the multi-modality anatomical atlas may include one or morereference images of a plurality of types of imaging modalities (e.g.,one or more reference CT images of a torso, one or more reference PETimages of a torso, one or more reference MR images of a torso) and mayinclude one or more organ models. Each organ model may includeprobabilistic data of a single organ or a set of organs. In some cases,the organ models are derived from medical images from designated imagingmodalities. For example, the multi-modality anatomical atlas may includean organ model that is derived from medical images obtained by a CTimaging system and an organ model that is derived from medical imagesobtained by a PET imaging system. In some embodiments, an organ modelmay be derived from medical images from multiple types of imagingmodalities.

The method 260 includes receiving at 262 first and second medical imagesof a region-of-interest (ROI) of a same individual. The first and secondmedical images are acquired by different first and second imagingmodalities. For example, the first and second medical images may be CTand PET images, respectively. In some embodiments, a third medical imageof the individual may be received that is acquired by a third medicalimaging modality (e.g., MR) that is different from the first and secondimaging modalities. The method 260 also includes generating at 264 firstand second feature images based on the first and second medical images.The first and second feature images include a same designated anatomicalfeature of the ROI (e.g., bone or soft tissue). The feature images maybe generated as described above. In such embodiments that include athird medical image, the generating at 264 may include generating athird feature image based on the third medical image.

The method 260 may also include determining at 266 a common identifiableportion or section in each of the first and second medical images anddetermining at 267 a transformation function by registering the firstand second feature images. The identifiable portion(s) may be, forexample, a particular cross-section of an anatomical structure (e.g.,organ) or other dimension of the anatomical structure, an internallandmark (e.g., particular bone structure or non-anatomical markerinserted into patient), surface contour of an anatomical structure, anexternal non-anatomical marker (e.g., stickers or stereotactic frames),and the like.

The determination at 267 may be similar to the determination at 222(FIG. 3) described above and may be based, at least in part, on theidentifiable portions determined at 266. For example, a commonidentifiable portion in each of the first and second feature images maybe used to register the first and second medical images. Rigid andnon-rigid transformation may be applied to the first and second featureimages.

The method 260 may also include applying at 268 the transformationfunction to the first and second medical images to register the medicalimages. At 270, a multi-modality anatomical atlas may be generated. Forinstance, an atlas generator of a computing system may assign the firstand second medical images and, optionally, the first and second featureimages to a multi-modality anatomical atlas. The first and secondmedical images may be used as reference images.

The atlas generator may also assign one or more organ models to theanatomical atlas. When the multi-modality anatomical atlas is initiallygenerated, the organ model(s) may be based on, for example,previously-developed organ models or other historical data. In someembodiments, the organ model(s) may be based on the first and secondmedical images. In such cases, when the anatomical atlas is initiallydeveloped, a first organ model may only be determined by the organ(s) inthe first medical image and a second organ model may only be determinedby the organ(s) in the second medical image. In other embodiments, asingle organ model may be derived from the first and second medicalimages.

The organ model may be subsequently updated one or more times. Forinstance, the organ model may be updated by the addition of a pluralityof determined organ placements. For example, each of ten CT images often different patients may be registered to a CT reference image todetermine organ placements of a designated organ in the ten CT images.The organ model may then be updated based on the organ placementsdetermined from the ten CT images.

The method 260 may continue as shown in FIG. 5B. In some embodiments,the method 260 may also include obtaining at 272 an input image of theROI from a different individual. The input image may be a medical imagethat is acquired by the first imaging modality. In other words, theinput image may be acquired by the same imaging modality as one of thereference images in the anatomical atlas. If the anatomical atlasincludes a suitable reference image (e.g., a reference image that isacquired through the same imaging modality as the input image), themethod 260 may also include determining at 273 identifiable portions ofthe input image and the reference image and determining at 274 atransformation function by registering the reference image with theinput image. In alternative embodiments, feature images may be generatedthat are based on the input image and the reference image. The method260 may also include applying at 276 the transformation function to theorgan model to generate a registered organ model. The registered organmodel may be used to obtain one or more organ placements at 277 of theorgan(s) in the ROI. The organ placement may then be used to update arespective organ model at 278.

In some embodiments, the method 260 may include obtaining at 280 aninput image of the ROI from a different individual in which the inputimage may be a medical image acquired by an imaging modality that isdifferent from the first and second imaging modalities. The remainingoperations may be similar to the registration and determinationoperations 206 and 208 described above. At 282, a third feature imagemay be generated that is based on the input image. The third featureimage may include the designated anatomical feature of the first andsecond feature images. Identifiable portion(s) in the third featureimage and the reference feature image (e.g., the first and/or secondfeature image) may be determined at 283. The method 260 may also includedetermining at 284 a transformation function by registering thereference feature image with the third feature image. At 286, thetransformation function may be applied to the organ model to generate aregistered organ model. The registered organ model may be applied at 287to the input image to determine one or more organ placements. Again, theorgan model may be updated at 278 with the organ placement.

FIG. 6 is a pictorial view of an exemplary imaging system 300 that isformed in accordance with various embodiments. FIG. 7 is a blockschematic diagram of a portion of the multi-modality imaging system 300shown in FIG. 6. The multi-modality system 300 may communicate with orinclude the computing system 100 (FIG. 1). In alternative embodiments,the imaging system 300 is a single modality imaging system. Asillustrated, the multi-modality imaging system 300 includes a CT imagingsystem 302 and a PET imaging system 304. However, as described above,other modalities may be implemented in other embodiments. Moreover, thecomputing system 100 is not required to communicate with a singleimaging system having multiple modalities. Instead, the computing system100 may communicate with multiple imaging systems that are of differentmodalities.

The CT imaging system 302 includes a CT gantry 310 and a PET gantry 311.The gantry 310 has an x-ray source 312 that projects a beam of x-raystoward a detector array 314 on the opposite side of the gantry 310. Thedetector array 314 includes a plurality of detector elements 316 thatare arranged in rows and channels that together sense the projectedx-rays that pass through an object, such as the subject 306. The imagingsystem 300 also includes a computing system 320 that receives theprojection data from the detector array 314 and processes the projectiondata to reconstruct an image of the subject 306. The computing system320 may include the same or similar components and features of thecomputing system 100 described above. In operation, user commands andparameters are used by the computing system 320 to provide controlsignals and information to reposition a motorized table 322. Morespecifically, the motorized table 322 is utilized to move the subject306 into and out of the gantry 310. Particularly, the table 322 moves atleast a portion of the subject 306 through a gantry opening 324 thatextends through the gantry 310.

The imaging system 300 also includes a system module 330 that isconfigured to implement various methods and algorithms described herein.The module 330 may be implemented as a piece of hardware that isinstalled in the computing system 320. Optionally, the module 330 may beimplemented as a set of instructions that are installed on the computingsystem 320. The set of instructions may be stand alone programs, may beincorporated as subroutines in an operating system installed on thecomputing system 320, may be functions in an installed software packageon the computing system 320, and the like.

The detector 314 includes a plurality of detector elements 316. Eachdetector element 316 produces an electrical signal, or output, thatrepresents the intensity of an impinging x-ray beam and hence allowsestimation of the attenuation of the beam as it passes through thesubject 306. During a scan to acquire the x-ray projection data, thegantry 310 and the components mounted thereon rotate about a center ofrotation 340. FIG. 7 shows only a single row of detector elements 316(i.e., a detector row). However, the multislice detector array 314includes a plurality of parallel detector rows of detector elements 316such that projection data corresponding to a plurality of slices can beacquired simultaneously during a scan.

Rotation of the gantry 310 and the operation of the x-ray source 312 aregoverned by a control mechanism 342. The control mechanism 342 includesan x-ray controller 344 that provides power and timing signals to thex-ray source 312 and a gantry motor controller 346 that controls therotational speed and position of the gantry 310. A data acquisitionsystem (DAS) 348 in the control mechanism 342 samples analog data fromdetector elements 316 and converts the data to digital signals forsubsequent processing. For example, the subsequent processing mayinclude utilizing the module 330 to implement the various methodsdescribed herein. An image reconstructor 350 receives the sampled anddigitized x-ray data from the DAS 348 and performs high-speed imagereconstruction. The reconstructed images are input to the computingsystem 320 that stores the image in a storage device 352. In theexemplary embodiment, the reconstructed images may include a series ofCT images 380 and a series of PET images 382. Optionally, the computingsystem 320 may receive the sampled and digitized x-ray data from the DAS348 and perform various methods described herein using the module 330.The computing system 320 also receives commands and scanning parametersfrom an operator via a console 360 that has a keyboard. An associatedoperator display 362 allows the operator to observe the reconstructedimage and other data from computer.

The operator inputs (e.g., supplied commands and parameters) are used bythe computing system 320 to provide control signals and information tothe DAS 348, the x-ray controller 344 and the gantry motor controller346. In addition, the computing system 320 operates a table motorcontroller 364 that controls the motorized table 322 to position thesubject 306 in the gantry 310. Particularly, the table 322 moves atleast a portion of the subject 306 through the gantry opening 324 asshown in FIG. 6.

Referring again to FIG. 7, in one embodiment, the computing system 320includes a device 370, for example, a non-transitory computer readablemedium such as, for example, a floppy disk drive, CD-ROM drive, a DVDdrive, a magnetic optical disk (MOD) device, or any other digital deviceincluding a network connecting device such as an Ethernet device forreading instructions and/or data from a computer-readable medium 372,such as a floppy disk, a CD-ROM, a DVD or an other digital source suchas a network or the Internet, as well as yet to be developed digitalmeans. In another embodiment, the computing system 320 executesinstructions stored in firmware (not shown). The computing system 320 isprogrammed to perform functions described herein, and as used herein,the term computer is not limited to just those integrated circuitsreferred to in the art as computers, but broadly refers to computers,processors, microcontrollers, microcomputers, programmable logiccontrollers, application specific integrated circuits, and otherprogrammable circuits, and these terms are used interchangeably herein.

In the exemplary embodiment, the x-ray source 312 and the detector array314 are rotated with the gantry 310 within the imaging plane and aroundthe subject 306 to be imaged such that the angle at which an x-ray beam374 intersects the subject 306 constantly changes. A group of x-rayattenuation measurements, i.e., projection data, from the detector array314 at one gantry angle is referred to as a “view”. A “scan” of thesubject 306 comprises a set of views made at different gantry angles, orview angles, during one revolution of the x-ray source 312 and thedetector 314. In a CT scan, the projection data is processed toreconstruct an image that corresponds to a two dimensional slice takenthrough the subject 306.

A technical effect of one or more embodiments described herein includesregistering an input image with one or more organ models that mayfacilitate further analysis and/or processing of the input image.Another technical effect of one or more embodiments may include a morefully automated computing system that requires less time and/or inputfrom a user of the computing system. Other technical effects may includeusing a reduced amount of computing resources as compared to knownmethods and/or improved imaging accuracy as compared to known methods.

As used herein, the term “computing system” may include anyprocessor-based or microprocessor-based systems including systems usingmicrocontrollers, reduced instruction set computers (RISC), applicationspecific integrated circuits (ASICs), logic circuits, and any othercircuit or processor capable of executing the functions describedherein. The above examples are exemplary only, and are thus not intendedto limit in any way the definition and/or meaning of the term “computingsystem.”

Sets of instructions may include various commands that instruct thecomputing system as a processing machine to perform specific operationssuch as the methods and processes described herein. The set ofinstructions may be in the form of a software program or module. Thesoftware may be in various forms such as system software or applicationsoftware. Further, the software may be in the form of a collection ofseparate programs, a program module (or module) within a larger program,or a portion of a program module. The software also may include modularprogramming in the form of object-oriented programming. The processingof input data by the processing machine may be in response to usercommands, or in response to results of previous processing, or inresponse to a request made by another processing machine. The program iscomplied to run on both 32-bit and 64-bit operating systems. A 32-bitoperating system like Windows XP™ can only use up to 3 GB bytes ofmemory, while a 64-bit operating system like Window's Vista™ can use asmany as 16 exabytes (16 billion GB).

As used herein, the terms “software” and “firmware” are interchangeable,and include any computer program stored in memory for execution by acomputing system, including RAM memory, ROM memory, EPROM memory, EEPROMmemory, and non-volatile RAM (NVRAM) memory. The above memory types areexemplary only, and are thus not limiting as to the types of memoryusable for storage of a computer program.

Accordingly, in one embodiment, a method of generating a multi-modalityanatomical atlas is provided. The method includes receiving first andsecond medical images of a region-of-interest (ROI) of a sameindividual. The first and second medical images are acquired bydifferent first and second imaging modalities. The method also includesgenerating first and second feature images based on the first and secondmedical images. The first and second feature images include a samedesignated anatomical feature of the ROI. The method also includesdetermining a transformation function by registering the first andsecond feature images and applying the transformation function to thefirst and second medical images to register the medical images. Themethod also includes generating a multi-modality anatomical atlas. Themulti-modality anatomical atlas has the first and second medical images.The first and second medical images are first and second referenceimages. The multi-modality anatomical atlas includes an organ model thatcorresponds to an organ in the ROI.

In another aspect, the method may also include obtaining an input imageof the ROI from a different individual. The input image is a medicalimage acquired by the first imaging modality. The method may alsoinclude determining another transformation function by registering thefirst reference image with the input image. The method may also includeapplying the other transformation function to the organ model togenerate a registered organ model and using the registered organ modelto obtain an organ placement of the organ in the ROI. The method mayalso include updating the organ model with the organ placement.

In another aspect, the method may include obtaining an input image ofthe ROI from a different individual. The input image may be a medicalimage acquired by a third imaging modality that is different from thefirst and second imaging modalities. The method may also includegenerating a third feature image based on the input image, wherein thethird feature image includes the designated anatomical feature of thefirst and second feature images. The method may also include determininganother transformation function by registering at least one of the firstfeature image or the second feature image with the third feature imageand applying the other transformation function to the organ model togenerate a registered organ model. The method may also include using theregistered organ model to obtain an organ placement of the organ withinthe ROI. The method may also include updating the organ model with theorgan placement.

It is to be understood that the above description is intended to beillustrative, and not restrictive. For example, the above-describedembodiments (and/or aspects thereof) may be used in combination witheach other. In addition, many modifications may be made to adapt aparticular situation or material to the teachings of the inventivesubject matter without departing from its scope. While the dimensionsand types of materials described herein are intended to define theparameters of various embodiments, they are by no means limiting and areonly example embodiments. Many other embodiments will be apparent tothose of skill in the art upon reviewing the above description. Thescope of the present application should, therefore, be determined withreference to the appended claims, along with the full scope ofequivalents to which such claims are entitled. In the appended claims,the terms “including” and “in which” are used as the plain-Englishequivalents of the respective terms “comprising” and “wherein.”Moreover, in the following claims, the terms “first,” “second,” and“third,” etc. are used merely as labels, and are not intended to imposenumerical requirements on their objects. Further, the limitations of thefollowing claims are not written in means-plus-function format and arenot intended to be interpreted based on 35 U.S.C. §112, sixth paragraph,unless and until such claim limitations expressly use the phrase “meansfor” followed by a statement of function void of further structure.

This written description uses examples to disclose the variousembodiments, including the best mode, and also to enable any personskilled in the art to practice the various embodiments, including makingand using any devices or systems and performing any incorporatedmethods. The patentable scope of the various embodiments is defined bythe claims, and may include other examples that occur to those skilledin the art. Such other examples are intended to be within the scope ofthe claims if the examples have structural elements that do not differfrom the literal language of the claims, or if the examples includeequivalent structural elements with insubstantial differences from theliteral languages of the claims.

What is claimed is:
 1. A method of generating a multi-modalityanatomical atlas, the method comprising: receiving first and secondmedical images of a region-of-interest (ROI) of a same individual, thefirst and second medical images being acquired by different first andsecond imaging modalities; generating first and second feature imagesbased on the first and second medical images, the first and secondfeature images including a same designated anatomical feature of theROI; determining a transformation function by registering the first andsecond feature images; applying the transformation function to the firstand second medical images to register the medical images; and generatinga multi-modality anatomical atlas, the multi-modality anatomical atlasincluding the first and second medical images, wherein the first andsecond medical images are first and second reference images, themulti-modality anatomical atlas including an organ model thatcorresponds to an organ in the ROI.
 2. The method of claim 1, furthercomprising: obtaining an input image of the ROI from a differentindividual, the input image being a medical image acquired by the firstimaging modality; determining another transformation function byregistering the first reference image with the input image; applying theother transformation function to the organ model to generate aregistered organ model and using the registered organ model to obtain anorgan placement of the organ in the ROI; and updating the organ modelwith the organ placement.
 3. The method of claim 1, further comprising:obtaining an input image of the ROI from a different individual, theinput image being a medical image acquired by a third imaging modalitythat is different from the first and second imaging modalities;generating a third feature image based on the input image, the thirdfeature image including the designated anatomical feature of the firstand second feature images; determining another transformation functionby registering at least one of the first feature image or the secondfeature image with the third feature image; applying the othertransformation function to the organ model to generate a registeredorgan model and using the registered organ model to obtain an organplacement of the organ within the ROI; and updating the organ model withthe organ placement.
 4. The method of claim 1, wherein the designatedanatomical feature is one of bone, soft tissue, or air that is within orsurrounds an anatomical structure.